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DETAILED ACTION 

Applicants' response to the last Office Action, filed on November 10*'', 2008 has been 
entered and made of record. 

Response to Arguments 

1 . Applicant's arguments filed 1 1/10/2008 have been fully considered but they are not 
persuasive. 

2. Applicant has argued on pages 2-3 that Trew in view of Hu has not taught a method that 
"models the aliasing effect" and goes into further detail regarding the non-claimed limitations, 
and where each is supported within the specification. However, neither the modeling of the 
aliasing effect nor the related details have been presented as limitations in the currently presented 
claims. Furthermore, in response to appUcant's argument that the references fail to show certain 
features of applicant's invention, it is noted that the features upon which applicant relies (i.e., 
"models the alias effects" and other spurious details) are not recited in the rejected claim(s). 
Although the claims are interpreted in light of the specification, limitations from the specification 
are not read into the claims. See In re Van Geims, 988 F.2d 1 181, 26 USPQ2d 1057 (Fed. Cir. 
1993). Additionally, as per the teachings of Trew in view of Hu have taught the Examiner's 
broadest reasonable interpretation of the currently presented claim language described in the 35 
use 103 claim rejections provided below. 



Response to Amendment 
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3. In response to the cancellation of claim 68, the 35 USC 1 12 rejection of claim 68 has 
been withdrawn. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

5. Claims 61, 67, 69-72, 90, and 98 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Trew (US 5280530) in further view of Hu (US 6483538). 

Instant claim 61 : A method for determining coordinates of a feature comprising: 

providing a first image including the feature, the first image comprising a plurality of 

pixels; [Trew has taught in column 2 line 20-37, column 3 lines 65+, and column 4 lines 20-65 
the use of an initial template image including the desired feature.] 

determining a first estimate of coordinates of the feature to within a fi-action of a pixel; 
[Trew further teaches in the sections referred to above the determination of the coordinates of a 
feature in a series of images, but does not teach a method of correlation wherein coordinates are 
determined with subpixel precision. However, Hu teaches in column 2 lines 31-65 the 
correlation of a pair of images obtained from a test video and captured video by correlation of 
the images. Furthermore, Hu teaches in column 3 lines 5-20 the measurement of a fractional 
pixel position and the shifting of the position to a "nearest integer pixel". Although, Hu does not 
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clearly state that the initial measurement is to a fraction of a pixel it is implied by the phrase 
"nearest integer pixel shift" which implies that a measurement of pixel position to a fraction of a 
pixel value was taken, else it would not be necessary to shift the pixel position to the "nearest 
integer pixel" and would simply be stated as an integer pixel shift. It would have been obvious to 
one of ordinary skill in the art at the time of the invention to modify the correlation method 
taught by Trew with the teachings of Hu to increase the accuracy of the positional measurement. 
Thus Trew in view ofHu has taught the subpixel positioning of a feature within an image using 
correlation.] 

translating the feature relative to the pixels [Regarding the new limitation: The feature 
described by Trew in view ofHu is described by pixels, and is shifted in relation to pixels.] by a 
pixel translation value, wherein the sum of the pixel fraction and pixel franslation value is an 
integer value; \Hu teaches in lines 5-20 of column 3 the shift of the measured pixel position to 
the nearest integer pixel position using the "nearest integer pixel position shift". Thus the shift 
referred to by Hu is the total shift value that translates the measured fractional pixel value to a 
"nearest integer pixel", but Hu does not explicitly teach the summation of a translation value to 
the fractional pixel position measured. However, in order to end up with a shift to a "nearest 
integer pixel" the fractional value needs to be mathematically manipulated in order to determine 
the nearest integer value. Examiner takes ojficial notice that one of ordinary skill in the art 
would have recognized that only a finite number of solutions exist for moving the fractional 
value to the nearest integer value. The known solutions are rounding the fractional value or 
adding (or subtracting "add a negative value") the difference value between the fractional value 
and its nearest integer value. Thus it has been established that Hu teaches shifting of the pixel 
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value to the "nearest integer" and that there are only a finite number of solutions for adjusting a 
fractional value to its closest integer value. A person of ordinary skill in the art at the time of the 
invention would have had good reason to pursue the known options of shifting to the closest 
integer value given the teachings of Hu. Furthermore, it would have required no more than 
"ordinary skill and common sense, " to sum the fractional value with a difference value that is 
equal to the distance between the fractional value and its closest integer value.] 

determining a second estimate of coordinates of the translated feature to within a fraction 
of a pixel; and \Hu teaches in column 3 lines 21-44 the determination of an additional fractional 
estimate of coordinates using the translated image.] 

summing the pixel fractions of the first estimate with the second estimate to derive a 
refined estimate of coordinates. [See Hu lines 30-32 of and 38-40 of column 3 (Xa+X/ = first 
estimate + second estimate).] 

Instant claim 90: A method for determining coordinates of an object, the method comprising the 
steps of 

capturing at least one first image and at least one second image of the object, each image 
being captured having different coordinates with respect to the other; [See the discussion of 
claim 61, wherein the capturing of a video sequence and the correlation of an image with a 
feature to at least one other image containing said feature by Trew in view of Hu. Furthermore, 
the teachings ofHu imply that each image has different coordinate with respect to each other 
since a specialized system of recording a sequence of images with exactly the same coordinates 
is not specified (very difficult to capture an image at exactly the same coordinate). Additionally, 
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the fact that the object has shifted in each image implies that the coordinate of each image is 
different.] 

determining the position of the object within each image, wherein determining includes; 
[See the discussion of claim 61.] 

providing the first image including a feature, the first image comprising a plurality of 
pixels; [See the discussion of claim 61.] 

determining a first estimate of coordinates of the feature to within a fi-action of a pixel; 
See the discussion of claim 61.] 

translating the feature relative to the pixels by a pixel translation value, wherein the sum 
of the pixel fraction and pixel translation value is an integer value; [See the discussion of claim 
61.] 

determining a second estimate of coordinates of the translated feature to within a fi-action 
of a pixel; [See the discussion of claim 61.] 

summing the pixel fi^actions of the first estimate with the second estimate to derive a 
refined estimate of coordinates; and [See the discussion of claim 61.] 

comparing the determined positions of the object to determine dimensional changes. 
Trew teaches the measurement of the displacement of the features in column 2 lines 30-40.] 

Instant claim 98: The method of claim 61, wherein the first image is captured with an image 
capture device. [As per the discussion of claim 61, both Hu (video test image) and Trew have 
taught the capturing of the first image with an image capture device.] 
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Instant claim 67: The method according to claim 61, wherein the translating step, second 
determining step and summing step are repeated at least once. [Hu in column 3 lines 20-45 
iterates the algorithm a specified number of iterations and checks the measured value with a 
noise value. '\ 

Instant claim 69: An apparatus for determining a position of an object comprising: 

an image capture device arranged to provide a captured image encompassing the object, 
the captured image comprising a plurality of pixels; and {Trew in lines 45-55 of column 5 
describes a camera for capturing the images containing the feature to be tracked.'] 

an image processor arranged to receive the captured image and determine the position of 
the object by executing the method of claim 61 . [Trew in lines 45-55 further describes a means 
for performing the processing, but does disclose the specific means. However, Hu in figure 1 
and column 2 lines 30-50 discloses a video processor for performing the video processing 
method.] 

Instant claim 70: The apparatus according to claim 69 further comprising: 

a monitor arranged to receive and display the captured image; and [Trew teaches a 

display means for displaying the images in column 6 lines 25-35.] 

an object selection means arranged to select a fiirther object within the displayed image 

and to identify the further object to the image processor. [Trew teaches in column 6 lines 55 the 

selection (tracing of feature to be used as template) of the template by automatic or manual 
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means. Furthermore, Tracing as taught by Trew must be done on a displayed image, and thus 
Trew teaches a selection means wherein the object is selected within the displayed image.] 

Instant claim 71 : An apparatus for determining a position of an object comprising: 

an image capture device arranged to sequentially provide a plurality of captured images 
of an object, each captured image comprising a plurality of pixels; and [Trew in lines 45-55 of 
column 5 describes a camera for capturing a series of images containing the feature to be 
tracked.] 

an image processor arranged to sequentially receive the plurality of captured images and 
determine the position of the object from the plurality of captured images by executing the 
method of claim 61; and [Trew in lines 45-55 further describes a means for performing the 
processing, but does disclose the specific means. However, Hu in figure 1 and column 2 lines 
30-50 discloses a video processor for performing the video processing method.] 

a position comparator arranged to compare the determined position of the object for the 
plurality of captured images and identify whether the determined position changes in the 
plurality of captured images. [Trew teaches the measurement of the displacement of the features 
in column 2 lines 30-40.] 

Instant claim 72: The apparatus according to claim 71 further arranged to determine the change 
in the determined position, the change selected from the group consisting of magnitude, 
direction, and combinations thereof. [Trew teaches the measurement of the displacement 
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(displacement is magnitude of positional change, but also includes directional information for 
tracking) of the features in column 2 lines 30-40.] 

6. Claims 91 and 99 are rejected under 35 U.S.C. 103(a) as being unpatentable over Trew in 
view of Hu as applied to claim 61 above, and further in view of Torre-Bueno et. al. (US 
2003/0231791). 2002/0044682 

Instant claim 91 : The method of claim 61, wherein the refined estimate of coordinates is 

recorded on a computer readable medium. [Trew in view ofHu have taught the detection of a 
feature using correlation, and the measurement of the shift and position in the image, but have 
not taught the store of the positional information. However, Torre-Bueno has taught in 
paragraph 0070 the measurement of an objects location in an image and the storage (on a 
computer readable medium) of the information for later using in processing the information. It 
would have been obvious to one of ordinary skill in the art to modify the teachings of Trew in 
view ofHu (object detection and subpixel positional measurement) with the teachings of Torre- 
Bueno to store the information for later use.] 

Instant claim 99: The method of claim 90, wherein the coordinates are recorded on a computer 
readable medium. {See the discussion of claim 91.] 



Application/Control Number: 10/532,438 Page 10 

Art Unit: 2624 

7. Claims 92-97 are rejected under 35 U.S.C. 103(a) as being unpatentable over Trew in 
view of Hu as applied to claim 61 and 90 above, and further in view of Well et. al. (US 
2002/0044682). 

Instant claim 92: The method of claim 90, further comprising determining a 2-dimensional 
position of the feature within the at least first image and the at least second image, wherein a 
position of the at least second image is known relative to the at least first image. [Trew in view 
ofHu has taught the positional determination of an object from one image in the other image 
(second image), but has not taught that the position of the at least second image is known related 
to the first image. However, Well has taught the positioning of stereoscopic images (necessary 
to know relative position in order to determine 3-D position of object captured by both imagers), 
and the correlation of a feature from one image to the other in order to determine the positioning 
of the object capture by the imagers (paragraph 0086). It would have been obvious to one of 
ordinary skill in the art to modify the positional determination method described by Well using 
the subpixel method taught by Trew in view ofHu to increase the accuracy of the positional 
information (3-D position includes 2-D position information).] 

Instant claim 93: The method of claim 92, fiirther comprising calculating a 3 -dimensional 
position of the feature from the 2-dimensional position for the at least two images. [See the 
discussion of claim 92.] 
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Instant claim 94: The method of claim 61, further comprising determining coordinates of the 
feature within a second image, the position of the second image being known relative to the first 
image. [See the discussion of claim 92.] 

Instant claim 95: The method of claim 61, further comprising determining a difference in 
position of the feature between the first image and at least one second image, wherein the at least 
one second image includes coordinates and has a position known relative to the first image. [See 
the discussion of claim 92, wherein the difference between the feature position is an inherent 
part of determining the 3-D position of the object due to parallax between the stereoscopic 
imaging devices.] 

Instant claim 96: The method of claim 61, fiirther comprising superimposing the first image and 

a second image to provide a superimposed image, wherein the position of the second image is 
known relative to the first image, and wherein the feature is substantially in registration. 
[Regarding the relative positioning, see the discussion of claim 92. Additionally, the method of 
correlation taught by Trew in view of Hu inherently includes the overlaying (superimposing) of 
an image with the feature over the other image and iteratively calculating the correlation value 
shifting until the feature is in registration (highest correlation value).] 

Instant claim 97: The method of claim 61, wherein the method is applied for monitoring an 
aircraft structure. [As per the discussion of claims 92-96, the use of a series of images to monitor 
and determine the position of an object has been taught, but the above cited prior art references 
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do not teach the monitoring of an aircraft structure. However, one of ordinary skill in the art at 
the time of the invention would have expected the disclosed system to be capable of detecting 
targeted structures, objects, or features in a plurality of environments and situations not 
explicitly listed by the cited prior art that require the tracking or positional determination of an 
object using a plurality of imagers that have been taught by the above cited art.] 



8. Claims 62-64, 66, and 77 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Trew in view of Hu as applied to claim 61 above, and further in view of Pankratov (US 
6208769). 



Instant claim 62: The method according to claim 61, wherein each of the first and second 
determining steps comprise: 

correlating the feature and the image using a predetermined correlation function to 
determine coordinates of the feature to the nearest pixel; [As per the rejection of claim 61 Trew 
in view ofHu has taught the use of a correlation function to determine the coordinates of a 
feature with sub-pixel accuracy.] 

evaluating the correlation function at a plurality of pixel positions in the neighborhood of 
the determined coordinates to provide a plurality of values; [Hu teaches in lines 35-45 of column 
3 the further correlation of the image to determine the fractional pixel shift, but does not limit the 
correlation to pixels in the neighborhood of the determined coordinates. However, in 
calculating the correlation values of the image as a whole Hu also teaches the evaluation of 
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correlation values within the neighborhood of the coordinates since they are included within the 
image being evaluated by the correlation function.^ 

fitting the plurality of values to a fijrther fiinction; and {Hu does not teach fitting the 
result (often referred to as score) of the correlation function to an equation or surface. 
However, as is evidenced by the teachings of Pankratov the fitting of correlation functions to 
surfaces for the purpose ofsubpixel measurement was known to one of ordinary skill in the art 
(see column 3 lines 32-52 of Pankratov). It would have been obvious to one of ordinary skill in 
the art at the time of the invention to modify the correlation method ofHu to more accurately 
determine the subpixel positioning using the fitting and determining method taught by 
Pankratov.'] 

differentiating the further fiinction to determine its turning point, whereby coordinates 
corresponding to the turning point provide coordinates of the feature. {Pankratov teaches the 

determination of the maximums (turning point) by differentiating the function and determining 
the points that make the functions equal to zero (see column 3 lines 42-45 of Pankratov).] 

Instant claim 63: The method according to claim 62, wherein the correlation fiinction is 
evaluated at a plurality of sub-pixel positions. {Pankratov teaches the evaluation of the function 
at regular intervals (column 5 lines 8-12 and figure 6 -points J, I, G, and H).] 

Instant claim 64: The method according to claim 63, wherein the sub-pixel positions are closer 
in proximity to the determined coordinates than the pixel positions. {See figure 6 (Pankratov) 
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points J, I, G, and H are closer to O (determined coordinates) than the pixel positions (points A, 
M,E. andD).] 

Instant claim 66: The method according to claims 62, wherein the predetermined correlation 
function is a normalized greyscale correlation function. [Pankratov teaches normalized 
correlation in lines 65+ of column 3. The normalized correlation is a shorthand reference to 
normalized greyscale correlation.] 

Instant claim 77: A method for determining coordinates of a feature comprising: 

providing at least one image including the feature, the at least one image comprising a 

plurality of pixels; [See the rejection of claim 61.] 

correlating the feature and the at least one image using a predetermined correlation 

function to determine coordinates of the feature to the nearest pixel; {See the rejection of claim 

62.] 

evaluating the correlation function at a plurality of sub-pixel positions in the 
neighborhood of the determined coordinates to provide a plurality of values and fitting the 
plurality of values to a further function; and {See the rejection of claims 62-63.] 

differentiating the further function to determine its maximum, whereby coordinates 
corresponding to the maximum are coordinates of the feature to within a fraction of a pixel. {See 
the rejection of claim 62 wherein the maxima occur at points when the differentiated function is 
zero (Pankratov).] 
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Conclusion 

9. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS fi-om the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

Contact Information 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Nathan Bloom whose telephone number is 571-272-9321 . The 
examiner can normally be reached on Monday through Friday from 8:30 am to 5:00 pm (EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Matthew Bella, can be reached on 571-272-7778. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be 
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obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

/Matthew C Bella/ 

Supervisory Patent Examiner, Art Unit 2624 



